Computational model for neural architecture search

A long-standing goal in Deep Learning (DL) research is to design efficient architectures for a given dataset that are both accurate and computationally inexpensive. At present, designing deep learning architectures for a real-world application requires both human expertise and considerable effort as they are either handcrafted by careful experimentation or modified from a handful of existing models. This method is inefficient as the process of architecture design is highly time-consuming and computationally expensive. The research presents an approach to automate the process of deep learning architecture design through a modeling procedure. In particular, it first introduces a framework that treats the deep learning architecture design problem as a systems architecting problem. The framework provides the ability to utilize novel and intuitive search spaces to find efficient architectures using evolutionary methodologies. Secondly, it uses a parameter sharing approach to speed up the search process and explores its limitations with search space. Lastly, it introduces a multi-objective approach to facilitate architecture design based on hardware constraints that are often associated with real-world deployment. From the modeling perspective, instead of designing and staging explicit algorithms to process images/sentences, the contribution lies in the design of hybrid architectures that use the deep learning literature developed so far. This approach enjoys the benefit of a single problem formulation to perform end-to-end training and architecture design with limited computational resources --Abstract, page iii

DenseNet for Anatomical Brain Segmentation

Automated segmentation in brain magnetic resonance image (MRI) plays an important role in the analysis of many diseases and conditions. In this paper, we present a new architecture to perform MR image brain segmentation (MRI) into a number of classes based on type of tissue. Recent work has shown that convolutional neural networks (DenseNet) can be substantially more accurate with less number of parameters if each layer in the network is connected with every other layer in a feed forward fashion. We embrace this idea and generate new architecture that can assign each pixel/voxel in an MR image of the brain to its corresponding anatomical region. To benchmark our model, we used the dataset provided by the IBSR 2(Internet Brain Segmentation Repository), which consists of 18 manually segmented MR images of the brain. To our knowledge, our approach is the first to use DenseNet to perform anatomical segmentation of the whole brain

Analysis of Parkinson\u27s Disease Data

In this paper, we investigate the diagnostic data from patients suffering with Parkinson\u27s disease (PD) and design classification/prediction model to simplify the diagnosis. The main aim of this research is to open possibilities to be able to apply deep learning algorithms to help better understand and diagnose the disease. To our knowledge, the capabilities of deep learning algorithms have not yet been completely utilized in the field of Parkinson\u27s research and we believe that by having an in-depth understanding of data, we can create a platform to apply different algorithms to automate the Parkinson\u27s Disease diagnosis to certain extent. We use Parkinson\u27s Progression Markers Initiative (PPMI) dataset provided by Michael J. Fox Foundation to perform our analysis

Efficient Architecture Search for Deep Neural Networks

This paper addresses the scalability challenge of automatic deep neural architecture search by implementing a parameter sharing approach with regularized genetic algorithm (RGE). The key idea is to use a regularized genetic algorithm (RGE) on a pre-determined template and discover a high-performance architecture by searching for the optimal chromosome. During evolution, each model corresponding to a discovered chromosome is trained for a fixed number of epochs to minimize a canonical cross-entropy loss on a given training dataset. Meanwhile, the performance of the trained model on validation dataset is used as a fitness value to perform the evolutions. Because of parameter sharing the trained weights in each generation are carried to the next, thereby reducing the GPU hours required for maximizing the validation accuracy. On the CIFAR-10 dataset, the approach finds a novel architecture that outperforms the best human-invented deep architecture (DenseNet). The CIFAR-10 model achieved a test error of 4.22% with only 0.96M parameters which is better than DenseNet of 4.51% with 0.8M parameters. On CIFAR-100 dataset, the approach was able to compose a novel architecture that achieved 20.53% test error with 3.7M parameters which is on par with 20.50% test error of wide ResNet with 36.5M parameters

System Architecting Approach for Designing Deep Learning Models

Deep Learning (DL) models have proven to be very effective in solving many challenging problems, especially, those related to computer vision, text, and speech. However, the design of such models is challenging because of the vast search space and computational complexity that needs to be explored. Our goal in this paper is to reduce the human effort required to design architectures by using a system architecture development process that allows the exploration of large design space by automating certain model construction, alternative generation, and assessment. The proposed framework is generic and targeted at all deep learning architectures that can be expressed by logical models with certain numeric properties. The implementation of the proposed approach is presented, along with the test results achieved on CIFAR-10 dataset using a convolutional neural network (CNN). We show that the architecture generated by our approach achieves 5.23% error rate with only 1.2M parameters, which shows the capability to design high performing architectures

Entity Resolution using Convolutional Neural Network

Entity resolution is an important application in field of data cleaning. Standard approaches like deterministic methods and probabilistic methods are generally used for this purpose. Many new approaches using single layer perceptron, crowdsourcing etc. are developed to improve the efficiency and also to reduce the time of entity resolution. The approaches used for this purpose also depend on the type of dataset, labeled or unlabeled. This paper presents a new method for labeled data which uses single layered convolutional neural network to perform entity resolution. It also describes how crowdsourcing can be used with the output of the convolutional neural network to further improve the accuracy of the approach while minimizing the cost of crowdsourcing. The paper also discusses the data pre-processing steps used for training the convolutional neural network. Finally it describes the airplane sensor dataset which is used for demonstration of this approach and then shows the experimental results achieved using convolutional neural network

Flexible and Intelligent Learning Architectures for SOS (FILA-SoS)

Multi-faceted systems of the future will entail complex logic and reasoning with many levels of reasoning in intricate arrangement. The organization of these systems involves a web of connections and demonstrates self-driven adaptability. They are designed for autonomy and may exhibit emergent behavior that can be visualized. Our quest continues to handle complexities, design and operate these systems. The challenge in Complex Adaptive Systems design is to design an organized complexity that will allow a system to achieve its goals. This report attempts to push the boundaries of research in complexity, by identifying challenges and opportunities. Complex adaptive system-of-systems (CASoS) approach is developed to handle this huge uncertainty in socio-technical systems